Ophthalmic apparatus and ophthalmic method

ABSTRACT

An ophthalmic apparatus includes a fundus image acquisition unit which acquires a plurality of fundus images acquired by imaging fundus of a subject&#39;s eye at different times and at least one fundus image that is fewer than the plurality of fundus images and acquired by imaging fundus of the subject&#39;s eye at a different time from those for the plurality of fundus images, a unit which generates a new fundus image by averaging the plurality of fundus images, an extraction unit which extracts a feature region from the generated new fundus image, and a unit which tracks the fundus such that positions of a first polarization tomographic image of the fundus corresponding to the new fundus image and a second polarization tomographic image of the fundus corresponding to the at least one fundus image may be corrected based on the extracted feature region and the at least one fundus image.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an ophthalmic apparatus and anophthalmic method.

2. Description of the Related Art

An optical coherence tomography (OCT) using a multiple-wavelength lightwave interference can acquire a high resolution tomographic image of asample (fundus in particular).

In recent years, an ophthalmologic OCT apparatus can acquire not only anormal OCT image in which the shape of a fundus tissue is captured butalso a polarization OCT image captured using a polarization parameter(retardation and orientation), which is one of optical characteristicsof the fundus tissue.

The polarization OCT can configure the polarization OCT image using thepolarization parameter, and can perform distinction and segmentation ofthe fundus tissue. The polarization OCT uses light modulated intocircularly polarized light as measuring beam for observing the sample todetect interfering light split as two orthogonal linear polarizationsand generate the polarization OCT image (refer to International PatentApplication WO2010/122118A1). However, International Patent ApplicationWO2010/122118A1 discloses no method for improving the quality of apolarization OCT image.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, there is provided anophthalmic apparatus including a fundus image acquisition unitconfigured to acquire a plurality of fundus images acquired by imagingfundus of a subject's eye at different times and at least one fundusimage that is fewer than the plurality of fundus images and acquired byimaging fundus of the subject's eye at a different time from those forthe plurality of fundus images, a unit configured to generate a newfundus image by averaging the plurality of fundus images, an extractionunit configured to extract a feature region from the generated newfundus image, and a unit configured to track the fundus such thatpositions of a first polarization tomographic image of the funduscorresponding to the new fundus image and a second polarizationtomographic image of the fundus corresponding to the at least one fundusimage may be corrected on basis of the extracted feature region and theat least one fundus image.

The present invention may improve the quality of a polarization OCTimage.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating an overall configuration ofan image processing apparatus according to an exemplary embodiment.

FIGS. 2A to 2E illustrate examples of images generated by a signalprocessing unit 190.

FIG. 3 is a flow chart illustrating a processing flow according to theexemplary embodiment.

FIG. 4 is a schematic diagram illustrating an example of an imageanalysis unit.

FIG. 5 is a flow chart illustrating a flow of processing according tothe exemplary embodiment.

FIG. 6 is an example of display on a display screen of a display unit ofthe image processing apparatus according to the exemplary embodiment.

FIG. 7 is an example of display on the display screen of the displayunit of the image processing apparatus according to the exemplaryembodiment.

FIG. 8 is an example of display on the display screen of the displayunit of the image processing apparatus according to the exemplaryembodiment.

FIG. 9 is a flow chart illustrating a flow of processing according tothe exemplary embodiment.

FIG. 10 is a flow chart illustrating a flow of processing according tothe exemplary embodiment.

FIG. 11 is a flow chart illustrating a flow of processing according tothe exemplary embodiment.

FIG. 12 is a diagram illustrating designation of a template position fortracking according to the exemplary embodiment.

FIG. 13 is a diagram illustrating calculation of a moving amount offundus according to the exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

An imaging apparatus according to the present invention is applicable tosubjects such as a subject's eye, a skin, and internal organs. Theimaging apparatus according to the present invention is anophthalmologic apparatus and an endoscope, for example. The followingdescribes in detail an ophthalmologic apparatus according to the presentexemplary embodiment as an example of the present invention withreference to the attached drawings.

[Overall Configuration of the Apparatus]

FIG. 1 is a schematic diagram illustrating an overall configuration ofan “ophthalmologic apparatus” which is an example of an imagingapparatus in the present exemplary embodiment. At least a part of asignal processing unit 190 described below can be regarded as an “imageprocessing apparatus.” In this case, the entire “ophthalmologicapparatus” can be regarded as an “ophthalmologic system”, or the entire“imaging apparatus” can be regarded as an “imaging system”.

The ophthalmologic apparatus includes a polarization sensitive OCT 100(hereinafter referred to as PS-OCT), a polarization sensitive scanninglaser ophthalmoscope 140 (hereinafter referred to as PS-SLO), ananterior eye imaging unit 160, an internal fixation lamp 170, and acontrol unit 200.

Alignment of the ophthalmologic apparatus is performed using an image atan anterior eye portion of the subject observed by the anterior eyeimaging unit 160 with the internal fixation lamp 170 turned on and thesubject's eye caused to gaze thereat. After the alignment is completed,the PS-OCT 100 and the PS-SLO 140 perform imaging of the fundus.

<Configuration of OCT 100>

The configuration of the OCT 100 is described.

A light source 101 is a super luminescent diode (SLD) light source lowin coherence and emits light with a center wavelength of 850 nm and aband width of 50 nm, for example. Although the SLD is used as the lightsource 101, any light source which can emit low-coherent light such asan amplified spontaneous emission (ASE) light source, for example, maybe used.

The light emitted from the light source 101 is guided to a fiber coupler104 with a polarization holding function via a polarization maintaining(PM) fiber 102 and a polarization controller 103, and split into ameasuring beam (hereinafter referred to as “measuring beam for atomographic image” or “OCT measuring beam”) and a reference beamcorresponding to the measuring beam.

The polarization controller 103 is for adjusting the state of lightemitted from the light source 101 to linear polarization. The branchingratio of the fiber coupler 104 is 90 (the reference beam) to 10 (themeasuring beam).

The measuring beam is emitted as parallel light from a collimator 106via a PM fiber 105. The emitted measuring beam reaches a dichroic mirror111 via an X scanner 107 including a galvanometer mirror which scans themeasuring beam in the horizontal direction at the fundus Er and a Yscanner 110 including a galvanometer mirror which scans the measuringbeam in the vertical direction at lenses 108 and 109 and the fundus Er.The X and Y scanners 107 and 110 are controlled by a drive control unit180 and can scan the measuring beam in a desired range of the fundus Er.The range in which the measuring beam is scanned on the fundus can beregarded as the acquisition range of the tomographic image, theacquisition position of the tomographic image, and the irradiationposition of the measuring beam. The X and Y scanners 107 and 110 areexamples of scanning unit for the PS-OCT and may be configured as acommon XY scanner. The dichroic mirror 111 has a characteristic thatreflects light with a wavelength of 800 nm to 900 nm and transmits therest.

The measuring beam reflected by the dichroic mirror 111 passes through aλ/4 polarizing plate which is arranged with an angle of 45° tiled from Ppolarization to S polarization with an optical axis as a rotation axisvia a lens 112 to shift the phase of the measuring beam by 90°, and ispolarization controlled to a circularly polarized light. The λ/4polarizing plate is an example of a polarization adjustment member forthe measuring beam for adjusting the polarization state of the measuringbeam. If the PS-SLO optical system described below is applied, the λ/4polarizing plate 113 can be provided in a common optical path of a partof the PS-OCT optical system and a part of the PS-SLO optical system.This allows comparatively suppressing the dispersion of polarizationstate occurring on the image acquired by the PS-SLO optical system andthe image acquired by the PS-OCT optical system. At this point, thescanning units for the PS-SLO and the PS-OCT are provided in positionsconjugate to each other and can be provided in positions conjugate tothe pupil of the subject's eye. The tilt of the λ/4 polarizing plate 113is an example of state of the λ/4 polarizing plate 113 and an angle froma predetermined position with the optical axis of a polarization splitface of a fiber coupler 123 incorporating a polarization beam splitteras a rotation axis, for example.

The measuring beam reflected by the dichroic mirror 111 passes through aλ/4 polarizing plate which is arranged with an angle of 45° tiled from Ppolarization to S polarization with an optical axis as a rotation axisvia a lens 112 to shift the phase of the measuring beam by 90°, and ispolarization controlled to a circularly polarized light. The λ/4polarizing plate is an example of a polarization adjustment member forthe measuring beam for adjusting the polarization state of the measuringbeam. If the PS-SLO optical system described below is applied, the λ/4polarizing plate 113 can be provided in a common optical path of a partof the PS-OCT optical system and a part of the PS-SLO optical system.This allows comparatively suppressing the dispersion of polarizationstate occurring on the image acquired by the PS-SLO optical system andthe image acquired by the PS-OCT optical system. At this point, thescanning units for the PS-SLO and the PS-OCT are provided in positionsconjugate to each other and can be provided in positions conjugate tothe pupil of the subject's eye. The tilt of the λ/4 polarizing plate 113is an example of state of the λ/4 polarizing plate 113 and an angle froma predetermined position with the optical axis of a polarization splitface of a fiber coupler 123 incorporating a polarization beam splitteras a rotation axis, for example.

The light incident on the subject's eye is polarization-controlled to acircularly polarized light by arranging the λ/4 polarizing plate tiltedby 45°, however, the light may not be controlled to the circularlypolarized light at the fundus Er depending on the characteristics of thesubject's eye. For that reason, the tilt of the λ/4 polarizing plate canbe fine-adjusted by controlling the drive control unit 180.

The measuring beam which is polarization-controlled to a circularlypolarized light is focused on the retinal layer of the fundus Er via theanterior eye portion Ea of eye, which is the subject, by a focus lens114 mounted on a stage 116. The measuring beam with which the fundus Eris irradiated is reflected and scattered by each retinal layer andreturns to the fiber coupler 104 via the above optical path.

The reference beam branched by the fiber coupler 104 is emitted as aparallel light from a collimator 118 via a PM fiber 117. As is the casewith the measuring beam, the emitted reference beam ispolarization-controlled by a λ/4 polarizing plate 119 which is arrangedat an angle of 22.5° tiled from P polarization to S polarization with anoptical axis as a rotation axis. The λ/4 polarizing plate 119 is anexample of a polarization adjustment member for the reference beam foradjusting the polarization state of the reference beam. The referencebeam passes through a dispersion compensation glass 120, is reflected bya mirror 122 on a coherence gate stage 121, and returns to the fibercoupler 104. This means that the reference beam passes through the λ/4polarizing plate 119 twice to cause linear polarization light to returnto the fiber coupler 104.

The coherence gate stage 121 is controlled by a drive control unit 180to cope with difference in eye's axial length. A coherence gate refersto a position corresponding to an optical path length of the referencebeam in the optical path of the measuring beam. In the present exemplaryembodiment, the optical path length of the reference beam is changed,however, a difference in an optical path length between the opticalpaths of the measurement and the reference beam has only to be changed.

The light returning to the fiber coupler 104 is combined with thereference beam to be interfering light (hereinafter referred also to as“combined light”). The interfering light is incident on the fibercoupler 123 incorporating the polarization beam splitter. Thepolarization beam splitter splits the interfering light into P and Spolarized light which are different in a polarization direction with asplit ratio of 50:50.

The P polarized light passes through a PM fiber 124 and a collimator130, is separated by a grating 131, and received by a lens 132 and aline camera 133. Similarly, the S polarized light passes through a PMfiber 125 and a collimator 126, is separated by a grating 127, andreceived by a lens 128 and a line camera 129. The gratings 127 and 131and the line cameras 129 and 133 are arranged in the direction accordingto each polarized light.

The light received by each of the line cameras 129 and 133 is output asan electric signal according to the strength of the light and receivedby a signal processing unit 190 being an example of a tomographic imagegeneration unit.

The tilts of the λ/4 polarizing plates 113 and 119 can be automaticallyadjusted based on the tilt of polarization split face of thepolarization beam splitter, but may be automatically adjusted withreference to a straight line connecting the center of optic disk of thefundus with the center of macula lutea of the fundus. It is desirable tohave a tilt detection unit (not illustrated) for detecting the tilts ofthe λ/4 polarizing plates 113 and 119. The tilt detection unit candetect a present tilt and a predetermined tilt. Needless to say, thetilts of the λ/4 polarizing plates 113 and 119 may be detected based onthe strength of the received light to adjust the tilts thereof so that apredetermined strength of light can be acquired. As described below, anobject indicating the tilt is displayed on a graphic user interface(GUI) and a user may adjust the tilt using a mouse. The polarizationbeam splitter and the λ/4 polarizing plates 113 and 119 are adjustedwith reference to the perpendicular direction as a polarizationreference to acquire the similar effect.

<Configuration of PS-SLO 140>

The configuration of the PS-SLO 140 is described below.

A light source 141 is a semiconductor laser and emits light whose centerwavelength is 780 nm, for example, in the present exemplary embodiment.The measuring beam emitted from the light source 141 (hereinafterreferred to as “measuring beam for fundus image” or “SLO measuringbeam”) passes through a PM fiber 142, is polarization controlled to alinearly polarized light by a polarization controller 145, and emittedas parallel light from a collimator 143. The emitted light passesthrough a hole portion of a perforated mirror 144 and reaches a dichroicmirror 154 via a lens 155, an X scanner 146 including a galvanometermirror which scans the measuring beam into the horizontal direction atthe fundus Er, lenses 147 and 148, and a Y scanner 149 including agalvanometer mirror which scans the measuring beam into the verticaldirection at the fundus Er. The X and Y scanners 146 and 149 arecontrolled by the drive control unit 180 and can scan a desired range onthe fundus using the measuring beam. The X and Y scanners 146 and 149are examples of the scanning unit for the PS-SLO and may be configuredas a common XY scanner. The dichroic mirror 154 has a characteristicthat reflects light with a wavelength of 760 nm to 800 nm and transmitsthe rest.

The measuring beam of liner polarization reflected by the dichroicmirror 154 passes through the same optical path as the measuring beam ofthe PS-OCT 100 and reaches the fundus Er.

The measuring beam with which the fundus Er is irradiated is reflectedand scattered at the fundus Er, and reaches the perforated mirror 144via the above-described optical path. The light reflected by theperforated mirror 144 passes through a lens 150, is split by apolarization beam splitter 151 into light different in polarizationdirection (P polarized light and S polarized light in the presentexemplary embodiment), received by avalanche photo diodes (APD) 152 and153, converted into a electric signal, and received by the signalprocessing unit 190, which an example of a fundus image generation unit.

The position of the perforated mirror 144 is conjugate to the positionof a pupil of the subject's eye. The light passing through the peripheryof the pupil among light in which the measuring beam with which thefundus Er is irradiated is reflected and scattered is reflected by theperforated mirror 144.

In the present exemplary embodiment, both of the PS-OCT and the PS-SLOuse the PM fiber, however, the use of a single mode fiber (SMF) allowsacquiring similar configuration and effect by controlling polarizationusing the polarization controller.

<Anterior Eye Imaging Unit 160>

The anterior eye imaging unit 160 is described.

In the anterior eye imaging unit 160, the anterior eye portion Ea isirradiated by an illumination light source 115 composed of lightemitting diodes (LED) 115-a and 115-b emitting illumination light with awavelength of 1000 nm. The light reflected by the anterior eye portionEa reaches a dichroic mirror 161 via the lens 114, the polarizing plate113, the lens 112, and the dichroic mirrors 111 and 154. The dichroicmirror 161 has a characteristic that reflects light with a wavelength of980 nm to 1100 nm and transmits the rest. The light reflected by thedichroic mirror 161 is received by an anterior eye camera 165 via lenses162, 163, and 164. The light received by the anterior eye camera 165 isconverted into an electric signal and received by the signal processingunit 190.

<Internal Fixation Lamp 170>

The internal fixation lamp 170 is described.

The internal fixation lamp 170 includes an internal fixation-lampdisplay unit 171 and a lens 172. A plurality of light emitting diodes(LD) arranged in a matrix form is used as the internal fixation-lampdisplay unit 171. A position where to turn on the light emitting diodesis changed according to a site desired to be imaged by controlling thedrive control unit 180. The light emitted from the internalfixation-lamp display unit 171 is guided to the subject's eye via thelens 172. The light emitted from the internal fixation-lamp display unit171 has a wavelength of 520 nm and a desired pattern is displayed by thedrive control unit 180.

<Control Unit 200>

The control unit 200 for controlling the entire apparatus is describedbelow.

The control unit 200 includes the drive control unit 180, the signalprocessing unit 190, a display control unit 191, and a display unit 192.

The drive control unit 180 controls each unit as described above.

The signal processing unit 190 includes an image generation unit 193 andan image analysis unit 194. The signal processing unit 190 generates animage, analyzes the generated image, and generates visualizationinformation of the analysis result based on the signals output from theline cameras 129 and 133, the APDs 152 and 153, and the anterior eyecamera 165. The generation and analysis of an image are described belowin detail.

The display control unit 191 displays on display unit 192 the imagesgenerated by the tomographic image generation unit and the fundus imagegeneration unit and acquired by a fundus image acquisition unit (notillustrated) and a tomographic image acquisition unit (not illustrated)respectively on a display screen. The display unit 192 is a liquidcrystal display, for example. The image data generated by the signalprocessing unit 190 may be transmitted by wire to the display controlunit 191 or by wireless. In this case, the display control unit 191 canbe regarded as an image processing apparatus. As an image system, thefundus image acquisition unit may include the SLO optical system, andthe tomographic image acquisition unit may include the OCT opticalsystem. In this specification, for the case of a subject except thesubject's eye, a “fundus image (fundus luminance image)” may beparaphrased in a “planar image (planar luminance image)” and the “fundusimage acquisition unit” may be paraphrased in a “planar imageacquisition unit”.

The display unit 192 displays display forms indicating various pieces ofinformation described below under the control of the display controlunit 191. The image data from the display control unit 191 may betransmitted by wire to the display unit 192 or by wireless. The displayunit 192 is included in the control unit 200, however, the presentinvention is not limited thereto, and the display unit 192 may beprovided separately from the control unit 200. Alternatively, a tabletmay be provided, which is an example of a user portable apparatus intowhich the display control unit 191 and the display unit 192 areintegrated. In this case, it is desirable to mount a touch panelfunction on the display unit to allow moving the display position of animage, expanding and reducing the image and changing the displayed imageon the touch panel.

[Image Processing]

The image generation by the image generation unit 193 included in thesignal processing unit 190 is described below.

The image generation unit 193 subjects the interference signals outputfrom the line cameras 129 and 133 to a re-configuration processing usedin a general spectral domain OCT (SD-OCT) to generate tomographic imagescorresponding to a first and a second polarized light which are twotomographic images based on each polarization component.

The image generation unit 193 reduces a fixed pattern noise from theinterference signal. The fixed pattern noise is reduced in such a mannerthat a plurality of the detected A scan signals is averaged to extractthe fixed pattern noise, and the extracted fixed pattern noise issubtracted from the input interference signal.

The image generation unit 193 converts the interference signal fromwavelength to the number of waves and performs Fourier-transform thereofto generate a tomographic signal indicating a polarization state.

The interference signal of two polarization components is subjected tothe above processing to generate two tomographic images.

The image generation unit 193 aligns the signals output from the APDs152 and 153 in synchronization with the drive of the X and Y scanners146 and 149 to generate fundus images corresponding to the first and thesecond polarized light which are two fundus images based on eachpolarization component.

<Generation Of Tomographic Luminance Image Or Fundus Luminance Image>

The image generation unit 193 generates the tomographic luminance imagefrom the two tomographic signals.

The tomographic luminance image is basically the same as the tomographicimage in the conventional OCT. The pixel value r is calculated from thetomographic signals A_(H) and A_(V) acquired from the line sensors 129and 133 by an equation 1:

$\begin{matrix}{r = \sqrt{A_{H}^{2} + A_{V}^{2}}} & (1)\end{matrix}$

Similarly, the image generation unit 193 generates the fundus luminanceimage from the two fundus luminance images.

FIG. 2A illustrates an example of a luminance image of an optic diskportion.

The display control unit 191 may cause the display unit 192 to displaythe tomographic luminance image acquired by a conventional OCT method onthe display unit 192 if the λ/4 polarizing plate 113 is removed from theoptical path or the fundus luminance image acquired by a conventionalSLO method on the display unit 192.

<Generation of Retardation Image>

The image generation unit 193 generates a retardation image from atomographic image of polarization components orthogonal to each other.

A value δ of each pixel of the retardation image is a value indicating aratio of influence of vertical and horizontal polarization components onthe subject's eye in a position of each pixel forming the tomographicimage. The value δ is calculated from the tomographic signals A_(H) andA_(V) by an equation 2:

$\begin{matrix}{\delta = {\arctan\left\lbrack \frac{A_{V}}{A_{H}} \right\rbrack}} & (2)\end{matrix}$

FIG. 2B illustrates an example of a retardation image of thus generatedoptic disk portion, and the retardation image can be obtained by theequation 2 for each B scan image. As described above, the retardationimage refers to the tomographic image indicating a difference ininfluence of two polarizations on the subject's eye. In FIG. 2B, thevalue indicating the ratio is displayed in color as the tomographicimage. Places where shading is thick are small in value indicating theratio. Places where shading is thin are large in value indicating theratio. For this reason, the generation of the retardation image allows abirefringent layer to be recognized. The details are as discussed in “E.Gotzinger et al., Opt. Express 13, 10217, 2005”.

Similarly, the signal processing unit 190 can also generate theretardation image in the planar direction of the fundus based on theoutput from the APDs 152 and 153.

<Generation Of Retardation Map>

The image generation unit 193 generates a retardation map from theretardation image acquired from a plurality of B scan images.

The image generation unit 193 detects a retinal pigment epithelium(hereinafter referred to as “RPE”) in each B scan image. The RPE has theproperty of dissolving polarization, so that the distribution ofretardation is examined along a depth direction without the range froman internal limiting membrane (hereinafter referred to as “ILM”) to theRPE by each A scan and the maximum value of the retardation is taken asthe representative value of the retardation in the A scan.

The image generation unit 193 performs above processing on allretardation images to generate a retardation map.

FIG. 2C illustrates an example of the retardation map of the optic diskportion. Places where shading is thick are small in value indicating theratio. Places where shading is thin are large in value indicating theratio. In the optic disk portion, a layer with birefringence is aretinal nerve fiber layer (hereinafter, referred also to as “RNFL”). Theretardation map is an image indicating a difference in influence whichtwo polarizations receive according to the birefringence of the RNFL andthe thickness of the RNFL. For this reason, the value indicating theratio is increased at a place where the RNFL is thick and decreased at aplace where the RNFL is thin. Therefore, with the retardation map, thethickness of the RNFL all over the fundus can be recognized, and theretardation map can be used for the diagnosis of glaucoma.

<Generation of Birefringence Map>

The image generation unit 193 linearly approximates the value of theretardation δ in the range from the ILM to the RNFL in each A scan imageof the previously generated retardation image and determines the tilt ofthe retardation δ as birefringence in a position on the retina of the Ascan image. In other words, the retardation is the product of distanceand birefringence in the RNFL, so that the values of depth andretardation in each A scan image are plotted to acquire a linearrelationship. Therefore, if the tilt is acquired by linearlyapproximating the plot using the least squares method, the tilt is avalue of the birefringence of the RNFL in the A scan image. All theretardation images are subjected to the processing to generate the mapindicating birefringence.

FIG. 2D illustrates an example of the birefringence map of the opticdisk portion. Because birefringence values are directly mapped, thebirefringence map can describe a fiber structure as a change inbirefringence if the fiber structure is changed even if the thickness ofthe RNFL is not changed.

<Generation of DOPU Image>

The image generation unit 193 calculates a Stokes vector S for eachpixel from the acquired tomographic signals A_(H) and A_(V) and a phasedifference Δφ between the tomographic signals using an equation 3.

$\begin{matrix}{s = {\begin{pmatrix}I \\Q \\U \\V\end{pmatrix} = \begin{pmatrix}{A_{H}^{2} + A_{V}^{2}} \\{A_{H}^{2} - A_{V}^{2}} \\{2\; A_{H}A_{V}\cos\;\Delta\;\phi} \\{2\; A_{H}A_{V}\sin\;\Delta\;\phi}\end{pmatrix}}} & (3)\end{matrix}$where, the phase difference Δφ is calculated from phases φ_(H) and φ_(V)of each signal acquired when two tomographic signals are calculated asΔφ=φ_(V)−φ_(H).

The image generation unit 193 sets a window with a size of about 70 μmin the main scanning direction of the measuring beam and about 18 μm inthe depth direction thereof in each B scan image, averages each elementof a stroke vector calculated for each pixel by a number C in eachwindow and calculates the degree of polarization uniformity (DOPU) ofpolarization in the window using an equation 4.DOPU=√{square root over (Q _(m) ² +U _(m) ² +V _(m) ²)}  (4)where, Q_(m), U_(m), and V_(m) are values in which the elements Q, U,and V of the stroke vector in each window are averaged. All the windowsin the B scan image are subjected to the processing to generate the DOPUimage of the optic disk portion illustrated in FIG. 2E. As describedabove, the DOPU image is the tomographic image indicating the degree ofuniformity of two polarizations.

The DOPU is a value indicating the degree of uniformity of polarizationand is the value near to 1 in a place where polarization is maintained,but is the value smaller than 1 in a place where polarization isdissolved and not maintained. In a structure of a retina, the RPE hasthe property of dissolving polarization, so that the value is smallerthan that in other areas in a portion corresponding to the RPE in theDOPU image. In FIGS. 2A to 2E, a place where shading is thin representsthe RPE and a place where shading is thick represents the area of theretinal layer where change is maintained. The DOPU image images a layercanceling polarization such as the RPE, so that the DOPU image can moresurely image the RPE than change in luminance even if the RPE isdeformed due to disease.

Similarly, the signal processing unit 190 can also generate the DOPUimage in the planar direction of the fundus based on the output from theAPDs 152 and 153.

In the present specification, the tomographic image, the retardationimage, and the DOPU image corresponding to the first and the secondpolarization described above are also referred to as a tomographic imagerepresenting the polarization state. Also in the present specification,the retardation map and the birefringence map described above are alsoreferred to as a fundus image representing the polarization state.

[Processing operation]

The processing operation of the image processing apparatus is describedbelow.

FIG. 3 is a flow chart illustrating the processing operation of theimage processing apparatus.

[Adjustment]

In step S101, the subject's eye is placed on the apparatus and theapparatus is aligned with the subject's eye. For the description ofalignment, only specific processing of the present exemplary embodimentis described. The descriptions of alignment in the XYZ directions suchas working distance, and the adjustment of focus and coherence gate areomitted because they are general.

<Capture Image>

A movement of a subject's eye due to an involuntary eye movement or poorfixation during PS-OCT imaging may distort a resulting PS-OCT image. Inorder to prevent this, a tracking operation which causes the movement ofan OCT scanner to follow the movement of the eye is performed duringPS-OCT imaging in step S102.

FIG. 10 is a detail flow chart illustrating a flow of step S102 (CaptureImage) according to the exemplary embodiment, and the processing in stepS102 will be described with reference to the flow chart in FIG. 10. FIG.10 illustrates steps S4010 to S4050 for implementing a trackingfunction. A tracking control unit 195 is responsible for the trackingfunction.

Step S4010 (Generate Template) in FIG. 10 follows the flow of processingillustrated in FIG. 11 in detail. In step S4010, a template image to beused for template matching is generated.

First, in step S5010 (Acquire SLO Fundus Luminance Image), the trackingcontrol unit 195 acquires 20 frames of generated SLO fundus luminanceimages from the APDs 152 and 153 for template generation.

As the number of frames of SLO fundus luminance images for templategeneration increases, the image quality of a template image generatedthrough image averaging performed in step S5040 improves, resulting inimproved accuracy of template matching. On the other hand, a highernumber of SLO fundus luminance images for template generation mayincrease the processing time for template generation. 20 or more framesmay be selected for the accuracy necessary for OCT photographing thoughthe number of frames may be determined in consideration of the necessaryaccuracy of matching and processing speed.

Next, in step S5020 (Designate Template Position), the tracking controlunit 195 displays the SLO fundus luminance images acquired in step S4010on the display unit 192 through the display control unit. An operatormay manually designate an image area to be used as a template by using amouse. FIG. 12 illustrates an example of a designated template position.FIG. 12 illustrates an SLO fundus luminance image in which the area AAcorresponds to a template-designated area.

An area having a characteristic pattern such as a position having adiverging blood vessel may be selected as the template-designated areaAA because it is used for pattern matching.

A plurality of template areas may be selected. When two or more templateareas are selected, a moving amount may be calculated with high accuracyby averaging shift amounts X and Y acquired as a result of templatematching.

The template position designation may be performed automatically by thetracking control unit 195. In this case, Harris Corner DetectionAlgorithm may be used as an algorithm for selecting an area having acharacteristic pattern.

Alternatively, Moravec Corner Detection Algorithm or other cornerdetection algorithms may be used as an algorithm for selecting an areahaving a characteristic pattern. Automatic selection of a plurality oftemplates may allow quick selection of a template position having acharacteristic pattern.

Next, in step S5030 (Image Alignment), the tracking control unit 195calculates registration error amounts of the 20 frames for templategeneration acquired in step S5010 and aligns the image patterns. Thetracking control unit 195 determines an image of one of the framesacquired in step S5010 as a template and performs template searchthrough the remaining 19 frames. More specifically, the tracking controlunit 195 calculates a Normalized Cross-Correlation (NCC) that is anindex representing a similarity at different positions on a templateimage, acquires a difference between image positions with a maximumindex value as a registration error amount, and aligns images of the 20frames.

The index representing a similarity may be any measure representing asimilarity of a characteristic between a template and an image within aframe. For example, Sum of Absolute Difference (SAD), Sum of SquaredDifference (SSD), Zero-means Normalized Cross-Correlation (ZNCC) or thelike may be used.

Next in step S5040 (Image Averaging), the tracking control unit 195averages the images aligned by step S5030 for the 20 frames of imagesacquired for template generation in step S5010.

Next, in step S5050, the tracking control unit 195 extracts an image onbasis of information on the template position selected in step S5020from the aligned and averaged images generated in step S5040. Theextracted image is a template image to be used for template matching.The template generation in step S4010 has been described in detail up tothis point.

In step S4020 (Save Template) in FIG. 10, the tracking control unit 195saves the template generated in step S4010 in a storage unit (notillustrated).

Next, in step S4030 (Acquire SLO fundus luminance image), the trackingcontrol unit 195 acquires one frame of an SLO fundus luminance image ora lower number of frames than the number of frames acquired in stepS5010 of an SLO fundus luminance image for calculation of a fundusmoving amount. A plurality of SLO images for calculating a fundus movingamount may be acquired, be superimposed and be averaged for higher imagequality. However, a higher number of SLO images may increase the periodof time from a time when an SLO fundus luminance image is first acquiredto the time when template matching completes, which may delay thetracking. A higher number of SLO images for calculating a fundus movingamount may reduce the frequency of template matching and may reduce thefrequency of OCT position correction (resulting in a lower operatingfrequency of tracking). From the viewpoint of prevention of trackingdelay and prevention of a lower operating frequency, the number of SLOimages for calculating a fundus moving amount is desirably lower thanthe number of frames for template generation acquired in step S5010.

A low-pass filter (LPF) is also applicable to SLO images for calculatinga fundus moving amount. A template image resulting from averaging of aplurality of images has a luminance value the fluctuations of which aresmoothed. Thus, an LPF may also be applied to SLO images for calculatinga fundus moving amount with fluctuations of luminance values smoothed sothat the corresponding template image and the SLO images for calculatinga fundus moving amount may have a substantially equal spatial frequencyfor higher accuracy of the template search in S4040. In this case, thefactor may be determined in accordance with the number of frames usedfor generating the template for a substantially equal frequency responseof the LPF.

Next, in step S4040 (Search Template), the tracking control unit 195searches the template image saved in step S4020 from the SLO fundusluminance images for calculating a fundus moving amount acquired in stepS4030 on basis of a pattern of a blood vessel, for example, rendered onthe image to calculate a fundus moving amount. FIG. 13 illustrates anSLO fundus luminance image for calculating a fundus moving amount. Afundus-moving-amount calculation method will be described with referenceto FIG. 13. An area AA′ is a template-designated position andcorresponds to the position of the area AA in FIG. 12. An area BB is aposition resulting from template search. An area CC is a differencebetween the positions of the areas AA′ and BB. The area CC is calculatedas a fundus moving amount.

The template search method is the same as the template search methodused in step S5030 (Image Alignment).

A similarity index with lower computational complexity may be acquiredin consideration of a tracking delay in step S4040.

The amount of rotations θ may be calculated as a fundus moving amount byadding the amounts of shifts X and Y.

Next, in step S4050 (Output Fundus Moving Amount to Driving ControlUnit), the tracking control unit 195 outputs the fundus moving amountacquired in step S4040 to the driving control unit 180.

The driving control unit 180 controls the X scanner 107 and Y scanner110 in accordance with two input X and Y moving amounts such that theOCT photographing position may be constant at all times. Thus,tomographic images resulting from capturing images at a same positionmay be acquired at all times even when the subject's eye moves due to aninvoluntary eye movement or poor fixation.

Next, in step S4060 (PS-OCT measurement), the control unit 200 causesthe light source 101 to emit measurement light and the line cameras 129and 133 to receive return light from the retina Er.

An operator may then determine whether to finish or continue the imagingin series. If the PS-OCT imaging is to be continued in series, thetracking control unit 195 returns to step S4030, and steps S4030 toS4060 are repeated.

For tracking of a movement of a subject's eye, the operating rate of thePS-SLO 140 may be faster than the operating rate of the PS-OCT 100. Forexample, in a case where the B scanning rate of PS-OCT is equal to 15 Hzand if the rate of acquisition and processing of SLO fundus luminanceimages with a PS-SLO is equal to 60 Hz, position correction may beperformed four times during one B scanning operation.

Position correction with a scanner may be performed when an image of acenter part of a photographed area is less distorted by a movement ofthe scanner. The scanning rate or correction timing may be adjusted onbasis of the acquisition rates of the PS-OCT and PS-SLO such that thescanner correction may avoid the center of the image.

In order to increase the processing speed, one output from two APDs mayonly be used instead of SLO fundus luminance images. The PS-SLO may usemeasurement light having a line shape to scan a fundus, instead of theconfiguration in FIG. 1. Adopting this configuration may eliminate thenecessity for scanning with the X scanner 146, which allows quickeracquisition of SLO fundus luminance images.

According to this embodiment, SLO fundus luminance images are used as animage used for tracking. An SLO luminance image may be acquired with asum of squares of p-polarized light and s-polarized light, andpolarization information on measurement light is not used. Thus, the λ/4polarizer 113 may be removed from the optical path for acquiring an SLOfundus luminance image for tracking.

According to this embodiment, the PS-SLO 140 includes the X scanner 146and Y scanner 149 as a galvano scanner to scan a desirable range on afundus with measurement light and avalanche photodiodes (APD) 152 and153 as light detecting units for measurement light. However, the Yscanner may only be used as the galvano scanner, and a line sensor maybe used as a light detecting unit for measurement light.

<Image Generation>

In step S102 for image capturing, a signal of return light from theretina Er is output to the image generating unit 193, and images aregenerated in the aforementioned manner.

<Analysis>

The luminance value of the tomographic image of ill eye can be lowerthan that of the tomographic image of healthy eye due to influence ofillness. A retinal layer may be overlooked or erroneously detected dueto the influence. For this reason, in step S104, the image analysis unit194 detects each layer of the retinal layer using information about aplace where the polarization state is randomized calculated by the imagegeneration unit 193 in step S103.

FIG. 4 illustrates a configuration of the image analysis unit 194. InFIG. 4, an image acquisition unit 2000 is an acquisition unit foracquiring the tomographic image indicating the polarization stategenerated by the image generation unit 193. The acquisition unit canalso be configured to acquire an image from an external server.

A difference image generation unit 3010 and an exudate extraction unit2020 are extraction units. The difference image generation unit 3010extracts the RPE as an example of a predetermined layer by analyzing thecontinuity of a polarization canceling material and subtracts the RPEfrom the tomographic image indicating the polarization state. Theexudate extraction unit 2020 subtracts the predetermined layer from theimage and then extracts the exudates.

An analysis unit 2030 acquires information about the position and sizeof the exudate acquired by the exudate extraction unit 2020. An outputunit 2040 outputs processing results to the display control unit 191.The analysis unit 2030 and the output unit 2040 may be included in thedisplay control unit 191.

FIG. 5 is a flowchart illustrating a detailed flow of the processing instep S104. The processing in step S104 is described in detail accordingto the flow of the processing in FIG. 5.

In step S2000, an image acquisition unit 2000 acquires a plurality ofthree-dimensional tomographic images each indicating a polarizationstate, which are imaged at different times. A DOPU image acquired bycalculating the DOPU using the equation 4 can detect the position of theRPE layer as the predetermined layer because the RPE cancels thepolarization state in the retinal layer. The RPE has a layer structure,so that the RPE exists as a mass with a certain capacity or more. On theother hand, exudates often scatter and are smaller than the layerstructure such as the RPE.

In step S2010, the difference image generation unit 3010 subjects thearea in a pixel value range in a predetermined range to reductionprocessing by using a filtering processing with a filter such as amorphological filter. For example, a dilation processing is performed.With this processing, exudates disappear. The difference imagegeneration unit 3010 enlarges the reduced image with a reverseprocessing. For example, an erosion processing is performed. The reverseprocessing refers to processing to expand an image by the amount equalto the reduction amount. This can provide the layer structure of theRPE. In steps S2030 and S2040, the difference image generation unit 3010subjects the layer structure of the RPE to binary processing, forexample, and stores the area with the predetermined value or more as thearea of the RPE layer.

In step S2050, the difference image generation unit 3010 subjects theenlarged image to difference processing with respect to the tomographicimage indicating the original polarization state. With this processing,in step S2060, a depolarized area except the RPE layer is acquired(extracted) as exudates. Each image in a depolarized area andinformation about an area such as the extracted exudates are storedwhile being associated with the tomographic image indicating thepolarization state. Information about imaging time including imagingdate and time is also associated with the tomographic image indicatingthe polarization state. For this reason, each image in the area wherethe polarization is randomized and information about an area such as theextracted exudates are associated with the imaging time and stored inthe storage unit.

The analysis unit 2030 acquires the coordinates of center of gravity inthe depolarized area such as exudates acquired from each image. Acircumscribed area of the depolarized area such as exudates is acquiredand a size of the depolarized area such as exudates is acquired as size.In step S2070, the coordinates of center of gravity in the depolarizedarea and a size of the depolarized area such as exudates are associatedwith images such as exudates, and information about the position isassociated with the imaging time and stored in the storage unit.

<Output>

Output processing performed in step S105 of the generated images andanalysis results is described below. In the output processing accordingto the present exemplary embodiment, information acquired in step S104is effectively displayed.

When the image generation unit 193 and the image analysis unit 194 inthe signal processing unit 190 finish the generation and analysis ofeach image, the display control unit 191 generates output informationbased on the results, outputs the output information to the display unit192 and displays the output information thereon. FIG. 6 is an example inwhich the display control unit 191 superimposes the image area of theextracted exudates on a two-dimensional tomographic image and displaysit on the display unit 192.

The analysis unit 2030 performs accumulation processing on thethree-dimensional tomographic image, aligns it with the fundus image,which is a planar image, and associates information about thedepolarized area such as the exudates extracted by the extraction unitwith the coordinates of the fundus image, and stores the information.The display control unit 191 causes the image in the depolarized area,which is stored in the storage unit and imaged at different times, tocorrespond to the coordinates of the fundus image, which is a planarimage, and displays the image on the display unit 192 as changing image.In this case, a color to be displayed for each corresponding imagingtime is changed and displayed on the display unit 192 to make it clearhow the depolarized area such as exudates is changed with time.Furthermore, the imaging time corresponding to a changing image isassociated therewith and displayed to make it clear how the depolarizedarea such as exudates is changed with time. Moreover, the position andsize of the center of gravity in the depolarized area such as exudatesfor each imaging time are displayed together to make it understandablehow the size and the position are changed with time. FIG. 7 illustratesan example in which an image in an area where the polarization imaged atdifferent imaging times is randomized is displayed correspondingly tothe coordinates of the planar image, and the position and size of thecenter of gravity for each imaging time are displayed together on thedisplay unit. In this case, the image is superimposed onto the fundusimage, which is a planar image and displayed make it easy to performcomparison with the fundus image.

The upper left chart in FIG. 8 illustrates the fundus image. The upperright chart therein illustrates an image in a depolarized area isdisplayed correspondingly to the coordinates of the planar image and achanging image in which the position and size of the center of gravityfor each imaging time are displayed together. The lower left chartillustrates a position image displaying the center of gravity as aposition of the depolarized area correspondingly to the coordinates ofthe planar image and imaging time. The lower right chart illustrates thetomographic image crossing the area E where the polarization israndomized such as macula and exudates in the upper left chart. Thedisplay control unit 191 displays the images having thus differentpieces of information alongside of each other on the display unit 192 toallow observing easily how the depolarized area such as exudates ischanged with time.

[Modification Example 1]

FIG. 9 is a flowchart illustrating a flow of processing in amodification example in which information about a depolarized area suchas exudates is extracted.

In step S3000, the image acquisition unit 2000 acquires a plurality ofthree dimensional tomographic images, which are imaged at differenttimes, each indicating the polarization state. In steps S3010 and S3020,an area extraction unit extracts information about a stored RPE layerarea, removes the information about the RPE layer area from each of thethree dimensional tomographic images to acquire a pixel value range inthe predetermined range as information about the depolarized area suchas exudates.

The analysis unit 2030 performs a processing for expanding and reducingthe depolarized area such as exudates acquired from each image. In stepsS3010 and S3020, the depolarized area such as exudates is extracted asan area with a certain degree of size by this processing. The analysisunit 2030 acquires the coordinates and the size of center of gravity inthis area and stores them in the storage unit. A circumscribed area ofthe depolarized area such as exudates is acquired and a size of thedepolarized area such as exudates is acquired as size. In step S3040,the coordinates of center of gravity in the depolarized area and a sizeof the depolarized area such as exudates are associated with images suchas exudates, and information about position is associated with theimaging time and stored in the storage unit. In step S3050, thegenerated images and the analysis results are output.

Other Embodiments

Embodiments of the present invention can also be realized by a computerof a system or apparatus that reads out and executes computer executableinstructions recorded on a storage medium (e.g., non-transitorycomputer-readable storage medium) to perform the functions of one ormore of the above-described embodiment(s) of the present invention, andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s). The computer may comprise one or more ofa central processing unit (CPU), micro processing unit (MPU), or othercircuitry, and may include a network of separate computers or separatecomputer processors. The computer executable instructions may beprovided to the computer, for example, from a network or the storagemedium. The storage medium may include, for example, one or more of ahard disk, a random-access memory (RAM), a read only memory (ROM), astorage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2013-005393, filed Jan. 16, 2013, and No. 2014-001524, filed Jan. 8,2014, which are hereby incorporated by reference herein in theirentirety.

What is claimed is:
 1. An ophthalmic apparatus, comprising: a fundusimage acquisition unit configured to acquire a plurality of fundusimages acquired by imaging fundus of a subject's eye at different timesand at least one fundus image that is fewer than the plurality of fundusimages and acquired by imaging fundus of the subject's eye at adifferent time from those for the plurality of fundus images; agenerating unit configured to generate a new fundus image by averagingthe plurality of fundus images; an extraction unit configured to extracta feature region from the generated new fundus image; a scanning unitconfigured to scan a measuring light in the fundus; and a control unitconfigured to control the scanning unit so as to correct, based on theextracted feature region and the at least one fundus image, positions oftomographic images in the fundus irradiated with the measuring light. 2.The ophthalmic apparatus according to claim 1, wherein the fundus imageacquisition unit acquires the plurality of fundus images and the atleast one fundus image at a frame rate equal to or higher than 60 Hz. 3.The ophthalmic apparatus according to claim 1, further comprising atomographic image acquisition unit configured to acquirepolarization-sensitive tomographic images as the tomographic imagesbased on a plurality of lights having polarization components differentfrom each other obtained by splitting a combined light obtained bycombining a light returning from the fundus irradiated with a measuringbeam via the scanning unit and a reference beam corresponding to themeasuring beam.
 4. The ophthalmic apparatus according to claim 3,wherein the tomographic image acquiring unit acquires thepolarization-sensitive tomographic images by using the measuring lightcontrolled so as to be scanned in same positions of the fundus.
 5. Theophthalmic apparatus according to claim 1, wherein the fundus imageacquisition unit is configured to acquire fundus luminance images of thesubject's eye as the fundus images based on a plurality of lights havingpolarization components different from each other obtained by splittinga light returning from the fundus irradiated with a light.
 6. Theophthalmic apparatus according to claim 1, wherein a number of frames ofthe plurality of fundus images is 20 or more.
 7. An ophthalmic method,comprising: acquiring a plurality of fundus images acquired by imagingfundus of a subject's eye at different times and at least one fundusimage that is fewer than the plurality of fundus images and acquired byimaging fundus of the subject's eye at a different time from those forthe plurality of fundus images; generating a new fundus image byaveraging the plurality of fundus images; extracting a feature regionfrom the generated new fundus image; scanning, by a scanning unit, ameasuring light in the fundus; and controlling the scanning unit so asto correct, based on the extracted feature region and the at least onefundus image, positions of tomographic images in the fundus irradiatedwith the measuring light.
 8. The ophthalmic method according to claim 7,wherein the acquiring acquires the plurality of fundus images and the atleast one fundus image at a frame rate equal to or higher than 60 Hz. 9.A non-transitory computer-readable storage medium configured to store aprogram causing a computer to execute the steps of the ophthalmic methodaccording to claim
 7. 10. An ophthalmic apparatus comprising: a scanningunit configured to scan a measuring light in a fundus of a subject'seye; a tomographic image acquisition unit configured to acquire, basedon a plurality of lights having polarization components different fromeach other obtained by splitting a combined light obtained by combininga light returning from the fundus irradiated with a measuring beam and areference beam corresponding to the measuring beam,polarization-sensitive tomographic images of the fundus, wherein the ofpolarization-sensitive tomographic images are imaged at different times;a fundus image acquisition unit configured to acquire a plurality offundus images of the subject's eye at a frame rate equal to or higherthan 60 Hz; and a control unit configured to control the scanning unitso as to correct, based on the plurality of fundus images, positions ofthe polarization-sensitive tomographic images in the fundus.
 11. Theophthalmic apparatus according to claim 10, wherein the tomographicimage acquiring unit acquires the polarization-sensitive tomographicimages by using the measuring light controlled so as to be scanned insame positions of the fundus.
 12. The ophthalmic apparatus according toclaim 10, wherein the fundus image acquisition unit is configured toacquire fundus luminance images of the subject's eye as the fundusimages based on a plurality of lights having polarization componentsdifferent from each other obtained by splitting a light returning fromthe fundus irradiated with a light.
 13. The ophthalmic apparatusaccording to claim 10, wherein a number of frames of the plurality offundus images is 20 or more.
 14. An ophthalmic method comprising:scanning, by a scanning unit, a measuring light in a fundus of asubject's eye; acquiring, based on a plurality of lights havingpolarization components different from each other obtained by splittinga combined light obtained by combining a light returning from the fundusirradiated with a measuring beam and a reference beam corresponding tothe measuring beam, polarization-sensitive tomographic images of thefundus, wherein the polarization-sensitive tomographic images are imagedat different times; acquiring a plurality of fundus images of thesubject's eye at a frame rate equal to or higher than 60 Hz; andcontrolling the scanning unit so as to correct, based on the pluralityof fundus images, positions of the polarization-sensitive tomographicimages in the fundus.
 15. A non-transitory computer-readable storagemedium configured to store a program causing a computer to execute thesteps of the ophthalmic method according to claim 14.